Skip to content

Document what physical_key schemes and timestamp tags actually mean - #818

Merged
fiskus merged 2 commits into
mainfrom
docs/physical-key-and-timestamp-tag-comments
Aug 4, 2026
Merged

Document what physical_key schemes and timestamp tags actually mean#818
fiskus merged 2 commits into
mainfrom
docs/physical-key-and-timestamp-tag-comments

Conversation

@fiskus

@fiskus fiskus commented Aug 4, 2026

Copy link
Copy Markdown
Member

Comments only — no behavior change. just lint clean, cargo doc -p quilt-rs clean.

These four comments record behavior that surprises, found while tracing whether a manifest row's physical_key scheme could serve as a local-vs-remote origin signal for a package-list recency view. It cannot, and none of the reasons were written down.

flow::install_paths

The doc comment claimed "installed paths have place pointing to file://location". They don't. Installed rows keep the remote s3:// key, and that's correct: physical_key names where the canonical bytes live, and keeping the remote key is what lets a later push skip the re-upload (matches_content ignores physical_key, so use_existing_row_or_upload copies the remote key across on a content match).

Three related fixes in that function:

  • the doc comment now describes actual behavior and why it's right;
  • the place value in the loop is marked diagnostic-only — it is computed, logged, and discarded (the error arm still works as an absolute-path assertion);
  • the plan sketch's "replace entry's physical key in the manifest with the cached physical key" step is marked as never implemented.

It also records the conclusion that motivated the trace: a file:// key means "these bytes were committed locally" — written by flow::commit and flow::create — and must not be read as a per-row origin marker.

flow::push::push_package_impl

The uploaded manifest is built into the cached manifests dir, and the success path never copies it over the installed one — copy_cached_to_installed runs only in the hash-mismatch error branch. So a locally committed row keeps its file:// key in .quilt/installed/ after a successful push, clearing only incidentally when a later pull or reset refreshes the installed manifest.

That divergence is safe because top_hash excludes physical_key (manifest::top_hasher hashes hash, logical_key, meta, size), so both copies are the same revision — which is also why the new_manifest_uri.hash != commit.hash guard doesn't fire on a normal push.

io::manifest::upload_row

The file:// scheme check is load-bearing, not defensive. Together with use_existing_row_or_upload's content-match branch it's what keeps a local key out of a published manifest: every row of an uploaded manifest goes through one of those two branches, so the key is either replaced with the S3 destination or came from the remote to begin with. It also closes the reverse leak — an already-s3:// row whose content doesn't match the destination hard-errors instead of publishing a manifest that addresses another bucket's objects.

io::manifest::tag_timestamp

.quilt/named_packages/<ns>/<epoch> is the registry's only record of revision times — the manifest format carries no timestamps in either the header or the rows. It's write-only from this client: reading revision times back means enumerating those tag objects, and the Remote trait has no list operation. Also noted next to the existing collision TODO that the tag key holds the commit time at one-second resolution, so two commits of the same namespace made within one second collide even if they're pushed hours apart — upload_tag's unconditional put means the second push silently overwrites the first's tag.


Review pass (force-pushed): corrected two inaccurate claims from the first version — file:// keys are written by create_package as well as commit, and the tag collision window is governed by commit time, not push time — dropped a non-exhaustive enumeration of Remote's methods, disambiguated the super::commit intra-doc link, and tightened all four comments (72 → 59 added lines).

🤖 Generated with Claude Code

Greptile Summary

This PR documents existing manifest-key and timestamp-tag behavior without changing executable behavior.

  • Clarifies that installed manifest rows preserve their existing physical_key.
  • Explains why successful pushes can leave installed and cached manifests with different physical-key schemes.
  • Documents upload-row scheme handling and timestamp-tag limitations.

Confidence Score: 5/5

The PR appears safe to merge.

No blocking failure remains.

Important Files Changed

Filename Overview
quilt-rs/src/flow/install_paths.rs Adds documentation explaining that installed rows retain their physical keys and that the computed file URL is diagnostic only.
quilt-rs/src/flow/push.rs Documents the cached-versus-installed manifest behavior after a successful push.
quilt-rs/src/io/manifest.rs Documents timestamp-tag semantics and the physical-key handling contract during row upload.

Reviews (2): Last reviewed commit: "Correct five inaccurate claims in the ph..." | Re-trigger Greptile

Context used:

Four comments recording behavior that surprises, found while tracing
whether a manifest row's `physical_key` scheme could serve as a
local-vs-remote origin signal for a UI recency view. It cannot, and the
reasons were not written down anywhere.

- `install_paths`: the doc comment claimed installed paths get a `place`
  pointing at `file://`. They do not — installed rows keep the remote
  `s3://` key, which is what lets a later push skip the re-upload via
  `matches_content`. The `place` value in the loop is computed and only
  logged, and the plan sketch's "replace entry's physical key" step was
  never implemented. Corrected all three, and recorded that a `file://`
  key means "committed locally" (written by `commit` and `create`) and
  must not be read as an origin marker.

- `push_package_impl`: the uploaded manifest is built into the *cached*
  manifests dir and never copied over the installed one on success, so
  `file://` rows outlive their push and clear only when a later pull or
  reset refreshes the installed manifest. Safe because `top_hash`
  excludes `physical_key`, so both copies are the same revision.

- `upload_row`: the `file://` scheme check is load-bearing, not
  defensive. With `use_existing_row_or_upload`'s content-match branch it
  is what keeps a local key out of a published manifest, and it also
  hard-errors rather than publishing a row that addresses another
  bucket's objects.

- `tag_timestamp`: `.quilt/named_packages/<ns>/<epoch>` is the registry's
  only record of revision times, since the manifest format carries none.
  It is write-only from this client — reading it back needs a list
  operation the `Remote` trait does not have. Also noted that the tag
  key holds the *commit* time at one-second resolution, so two commits
  made within one second collide even when pushed hours apart, the
  second push silently overwriting the first's tag.

Comments only; no behavior change.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@fiskus
fiskus force-pushed the docs/physical-key-and-timestamp-tag-comments branch from c605905 to c3a9243 Compare August 4, 2026 15:57
Review of c3a9243 found five comments whose stated mechanism does not
match the code:

- `install_paths` doc claimed retaining the remote `s3://` key is what
  lets a push skip the re-upload. It is not — `matches_content` compares
  logical_key/hash/size only, so dedup is identical with `file://` keys.
  Also dropped the "installed rows keep the remote `s3://` key" framing:
  this function copies rows verbatim, and the installed manifest it is
  handed carries `file://` for locally committed paths. Recorded the live
  consequence instead — the caching call parses `physical_key` as an
  `S3Uri`, so an uncached `file://` row errors rather than being fetched.

- `push_package_impl` claimed a later pull or reset clears the stale
  `file://` keys. Neither can: both short-circuit when `latest` is the
  hash just pushed. They clear only when another client publishes.

- `tag_timestamp`'s collision note read as if the second-pushed revision
  loses its history entry; it is the first.

- `upload_row`'s doc credited the `file://` scheme check with keeping
  local keys out of published manifests. That comes from the
  `remote_url` rewrite plus the content-match branch; the check guards
  the input, and for `s3://` keys `to_file_path` would fail anyway.

Comments only; no behavior change.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@fiskus

fiskus commented Aug 4, 2026

Copy link
Copy Markdown
Member Author

@greptileai please re-review — pushed e1d8c54, which corrects five comments whose stated mechanism didn't match the code (push dedup rationale, the "installed rows keep s3://" framing, the unreachable pull/reset healing path, an inverted antecedent in the tag-collision note, and the mis-attributed file:// scheme check). Still comments-only.

@fiskus
fiskus merged commit d607f28 into main Aug 4, 2026
3 checks passed
@fiskus
fiskus deleted the docs/physical-key-and-timestamp-tag-comments branch August 4, 2026 16:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant